An Empirical Study of Cost-sensitive Classification in Campaign Management

Extremely unbalanced data and unequal costs are key challenges in data mining for campaign management in CRM. This paper presents an empirical study of cost-sensitive classification in a real-world campaign analysis in the newspaper industry, where the response rate is extremely low while the two types of misclassification costs are very different. Incorporating cost information provided by domain experts, the authors investigate the use of three classification methods, i.e., naive Bayes, logistic regression, and decision tree, for campaign response prediction. The data are analyzed under various estimated cost matrices. Cross-validated evaluation results show that cost-sensitive classification techniques are capable of obtaining both higher expected response rates and higher expected profits than the current practice when the estimated misclassification costs are moderately unequal. This empirical study provides useful insights for practice in campaign management, as well as research in classification using extremely unbalanced data and unequal costs.